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PREFACE 


This  report  presents  the  provincial  results  of  the  English  33  Diploma 
Examination  administered  on  January  17  (Part  A)  and  January  27  (Part  B)  1984. 
During  this  first  administration,  the  English  33  Diploma  Examination  was 
written  by  4746  students.  This  report  is  intended  to  provide  more  information 
about  the  examination  development  process,  the  examination  itself,  and  the 
examination  results. 
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CHAPTER  1 


THE  GRADE  12  DIPLOMA  EXAMINATIONS  PROGRAM 


Introduction 

All  Grade  12  students  in  Alberta  are  now  required  to  write  at  least  ONE 
diploma  examination  in  order  to  receive  a high  school  diploma.  The  Grade  12 
Diploma  Examinations  Program,  which  is  an  integral  part  of  the  high  school 
diploma  requirements,  is  intended  to  develop  and  maintain  excellence  in 
educational  standards  through  certification  of  academic  achievement. 

The  Diploma  Examinations  Program  consists  of  course-specific  examinations  that 
are  based  on  the  prescribed  Program  of  Studies  for  Senior  High  Schools  for 
the  following  Grade  12  (30-level)  courses:  English  30,  English  33,  Social 

Studies  30,  Mathematics  30,  Biology  30,  Chemistry  30,  and  Physics  30. 

Alberta  Education  issues  two  distinct  high  school  diplomas:  the  General  High 

School  Diploma  and  the  Advanced  High  School  Diploma. 


Gene ral  High  School  Diploma 

To  earn  a General  High  School  Diploma,  a student  must  obtain  course  credit  in 
either  English  30  or  English  33,  and  obtain  100  credits  distributed  over 
courses  as  specified  in  the  Junior-Senior  High  School  Handbook.  Some 
students  who  are  working  toward  the  general  diploma  may  wish  to  obtain  credits 
in  other  diploma  examination  courses  (i.e..  Social  Studies  30,  Mathematics  30, 
Biology  30,  Chemistry  30,  and  Physics  30).  To  obtain  credits  in  these 
courses,  a student  must  also  write  the  appropriate  diploma  examination 
regardless  of  which  type  of  diploma  he  wishes  to  receive. 


Advanced  High  School  Diploma 

The  Advanced  High  School  Diploma  represents  achievement  in  an  academic  program 
that  includes  language  arts  (English),  social  studies,  mathematics,  and 
science.  To  earn  an  Advanced  High  School  Diploma,  a student  must  satisfy  the 
current  course  and  credit  requirements  for  a General  High  School  Diploma  and 
obtain  course  credit  in  English  30,  Social  Studies  30,  Mathematics  30,  and  ONE 
of  Biology  30,  Chemistry  30,  or  Physics  30. 


Awarding  of  Course  Credits 

To  obtain  credit  in  any  Grade  12  (30-level)  course,  a student  must  earn  a 
final  mark  of  50%  or  better.  To  obtain  credit  in  a Grade  12  (30-level) 
diploma  examination  course,  a student  must  write  the  appropriate  diploma 
examination  and  attain  a final  blended  mark  of  50%  or  better.  The  "final 
blended  mark"  is  made  up  of  50%  of  the  mark  awarded  by  the  school  and  50%  of 
the  diploma  examination  mark.  For  example,  a student  taking  English  33  might 
have  a mark  of  45%  from  his  school  and  a mark  of  57%  on  the  diploma 
examination.  This  student's  final  mark  is  the  average  of  the  two  marks,  or 
51%.  He  has  therefore  earned  credits  in  English  33  because  his  final  mark  is 
over  50%. 
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Transitional  Provisions 


During  the  1983-84  school  year,  Alberta  Education  will  recognize  all  course 
credits  earned  prior  to  September  1,  1983,  for  the  purpose  of  awarding  the 
General  High  School  Diploma. 

Students  who  have  completed  partial  requirements  for  the  Advanced  High  School 
Diploma  prior  to  September  1,  1983,  and  who  are  enrolled  in  Grade  12  courses 
during  the  1983-84  school  year,  may  apply  any  of  the  previously  completed 
required  diploma  examination  subjects  toward  a diploma,  provided  they  have 
earned  a final  course  mark  of  50%  or  better  in  each  subject. 


Award  of  Excellence 

When  a candidate  for  an  Advanced  High  School  Diploma  obtains  a final  average 
of  80%  or  higher  on  the  four  required  diploma  examination  courses  with  not 
less  than  65%  in  any  one  of  these  four  required  courses,  he  receives  an  Award 
of  Excellence.  This  Award  of  Excellence  is  noted  on  the  student’s  Advanced 
High  School  Diploma. 

When  a student  writes  two  or  three  of  the  diploma  examinations  in  the  sciences 
(i.e..  Biology  30,  Chemistry  30,  and  Physics  30),  the  highest  of  these  final 
course  marks  is  used  for  diploma  purposes  and  in  the  calculation  of  the 
average  for  the  Award  of  Excellence. 


CHAPTER  2 


EXAMINATION  DESIGN,  DEVELOPMENT,  AND  DESCRIPTION 


Examination  Design  and  Development 

There  were  four  stages  in  the  development  of  the  English  33  Diploma 
Examination:  preparation  of  curriculum  and  examination  specifications, 

examination  design,  development  and  field-testing  of  questions,  and  selection 
of  questions  for  the  final  examination. 

The  Curriculum  Branch  of  Alberta  Education  prepared  curriculum  specifications 
that  identified  the  major  content  areas,  the  specific  objectives  within  each 
area,  and  the  emphasis  each  is  to  receive  in  the  classroom.  The  curriculum 
specifications  were  distributed  to  all  school  jurisdictions  in  the  province  in 
a document  entitled  Diploma  Examinations : Curriculum  Specifications  for 

English  33  (July  1983).  The  Student  Evaluation  Branch  of  Alberta  Education 
selected,  from  the  prepared  curriculum  specifications,  a set  of  diploma 
examination  specifications  that  formed  the  basis  of  evaluation  given  the 
limitations  of  a paper-and-pencil  examination  design.  These  diploma 
examination  specifications  constitute  Section  C of  the  Curriculum 
Specifications  for  English  33. 

The  Student  Evaluation  Branch,  in  consultation  with  language  arts  supervisors 
and  consultants  and  with  classroom  teachers,  then  designed  the  English  33 
Diploma  Examination.  The  examination  design,  complete  with  blueprints, 
scoring  guides,  and  sample  questions,  was  distributed  to  all  school 
jurisdictions  in  the  province  in  Student  Evaluation  Branch  Bulletin:  Volume 

3,  Number  3:  Grade  12  Diploma  Examinations  Program:  English  33  (September 

1983). 

Under  the  supervision  of  the  Student  Evaluation  Branch,  examination  questions 
were  developed  by  English  33  teachers  from  all  parts  of  the  province,  field- 
tested,  and  revised.  Questions  were  designed  to  reflect  the  reading,  writing, 
and  viewing  objectives  outlined  in  the  diploma  examination  specifications. 

The  final  examination  was  constructed  from  those  questions  that  best  reflected 
curricular  intent  and  examination  design.  It  was  reviewed  by  a committee  of 
classroom  teachers  who  estimated  difficulty  levels  for  each  question  and 
suggested  revisions.  The  target  mean  for  the  English  33  Diploma  Examination 
was  set  at  65%.  The  final  draft  was  approved  by  an  Examination  Review 
Committee  consisting  of  representatives  from  several  stakeholder  groups  (two 
Grade  12  English  teachers  representing  The  Alberta  Teachers’  Association;  one 
member  each  from  the  Conference  of  Alberta  School  Superintendents,  the 
Universities  Co-ordinating  Council,  and  the  Council  of  Presidents,  Public 
Colleges  of  Alberta),  and  representatives  of  the  Curriculum  Branch,  the 
Student  Evaluation  Branch,  and  the  Regional  Offices  of  Alberta  Education.  The 
Examination  Review  Committee  checked  the  draft  for  content  validity,  accuracy, 
and  technical  merit.  Changes  to  the  examination  recommended  by  the  committee 
were  made. 
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Examination  Description 


The  English  33  Diploma  Examination  consisted  of  two  parts:  Part  A:  Written 

Response,  worth  50%  of  the  total  diploma  examination  score,  and  Part  B: 

Reading  (Multiple  Choice),  also  worth  50%  of  the  total  score. 

Part  A:  Written  Response 

Part  A:  Written  Response  consisted  of  three  sections.  Section  I: 

Personal  Response  to  Literature  required  the  student  to  read  a short  selection 
that  served  as  a stimulus  for  a personal  or  reflective  response.  It  was  worth 
50%  of  the  mark  awarded  for  Part  A:  Written  Response  or  25%  of  the  total  mark 

for  the  English  33  Diploma  Examination.  Section  II:  Functional  Writing 

required  the  student  to  nominate  someone  for  a citizenship  award  and  to 
convince  an  awards  committee  that  the  person  being  nominated  should  receive 
the  award.  This  section  was  worth  30%  of  the  mark  awarded  for  Part  A or  15% 
of  the  total  mark  for  the  examination.  Section  III:  Critical  Response  to 

Visual  Communication  required  the  student  to  answer  three  questions  about  a 
cartoon.  The  student's  answers  were  required  to  be  in  paragraph  form.  This 
section  was  worth  20%  of  the  mark  awarded  for  Part  A or  10%  of  the  total  mark 
for  the  examination. 

Students  were  allowed  to  use  a dictionary  and  a thesaurus.  In  sections  I and 
II,  suggestions  for  writing  were  given  and  space  was  provided  for  planning 
activities  and  for  rough  work.  Students  were  allotted  two  and  one-half  hours 
to  write  Part  A:  Written  Response. 


Reporting  Categories  for  Part  A:  Written  Response 


To  provide  information  about  student  writing  that  is  meaningful,  it  is 
necessary  to  examine  students'  responses  in  terms  of  writing  components  or 
factors  that  can  be  used  as  reporting  categories.  Factors  evaluated  in 
Section  I:  Personal  Response  to  Literature  were:  Thought  and  Detail  — the 

content  present  in  the  student's  writing,  the  concepts  and  ideas  presented, 
and  specific  detail  selected  to  support  these  ideas;  Organization  — the 
coherent  development  of  a unified  theme  or  thesis  through  an  appropriate 
method  of  organization;  Matters  of  Choice  — the  style  of  writing  used  by  the 
student,  choices  of  words  and  syntactic  structures  used  to  give  emphasis  and 
effectiveness  to  writing;  and  Matters  of  Convention  — the  mechanics  of 
writing;  conventions  of  spelling,  punctuation,  and  grammar.  Factors  evaluated 
in  Section  II:  Functional  Writing  were  Thought  and  Detail,  Matters  of  Choice, 

and  Matters  of  Convention.  Those  evaluated  in  Section  III:  Critical  Response 

to  Visual  Communication  were  Thought  and  Detail  and  Matters  of  Convention. 

The  design  of  Part  A:  Written  Response  is  given  in  Table  1 on  page  5. 
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Many  of  the  reading  concepts  present  in  the  English  33  Statement  of  Content  are  dealt  with  in  the  reporting  category  headed  THOUGHT  AND  DETAIL 
in  the  test  section  PERSONAL  RESPONSE  TO  LITERATURE.  The  student  has  the  opportunity,  while  writing  Part  A:  Written  Expression,  to  construct 
his  own  response  to  literature  and  deal  with  concepts  also  tested  in  Part  B:  Reading  (Multiple  Choice). 
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Table  1 

English  33  Diploma  Examination  Blueprint 
Part  A:  Written  Response 


Part  B:  Reading  (Multiple  Choice) 


Part  B:  Reading  (Multiple  Choice)  consisted  of  65  items  based  on  9 reading 

passages  or  sets  of  information.*  Information  on  the  numbers  and  types  of 
stimuli  on  Part  B is  presented  in  Table  2. 

Table  2 


Distribution  of  Passage  or  Stimulus  Types 


Passage  or  Stimulus 
Type 

Number  of  Passages  or  Number  of 
Sets  of  Information  Items 

Modern  Drama 

1 

9 

Fiction 

2 

15 

Mon-fiction 

2 

17 

Poetry 

2 

11 

Information  Sets 

2 

13 

Totals 

9 

65 

Students  were  allotted  two  hours  to  write  Part  B.  Use  of  a dictionary  and 
thesaurus  was  not  allowed. 

Reporting  Categories  for  Part  B:  Reading  (Multiple  Choice) 

Items  were  grouped  into  four  reporting  categories  or  subtests:  Meanings 

(25  items).  Relationships  of  Form  and  Content  (8  items).  Human  Experience  and 
Values  (19  items)  and  Life  Skills  (13  items).  Each  reporting  category 
required  a minimum  of  six  items  to  achieve  statistical  reliability.  Table  3 
on  page  7 shows  which  items  made  up  each  of  the  reporting  categories.  In 
addition.  Table  3 gives  the  proportion  of  the  total  examination  mark  assigned 
to  each  reporting  category  or  subtest. 

Taxonomic  Levels  for  Part  B:  Reading  (Multiple  Choice) 

A further  design  consideration  affecting  the  development  of  Part  B was 
taxonomic  level.  Items  were  classified  according  to  three  taxonomic  levels: 
Literal  Understanding  (11  items).  Inference  and  Application  (43  items),  and 
Evaluation  (11  items).  By  considering  taxonomic  level  when  developing  an 
examination,  the  Student  Evaluation  Branch  attempts  to  ensure  that  a variety 
of  mental  activities  will  be  used  by  students  as  they  write  the  examination. 
Items  listed  under  Literal  Understanding  are  expected  to  be  answered  using 
skills  of  recognition  and  recall;  those  listed  under  Inference  and  Application 
are  expected  to  elicite  skills  of  analysis,  interpretation,  synthesis,  and 
application  of  knowledge;  and  items  listed  under  Evaluation  are  expected  to 
draw  forth  judgmental  skills. 


*Sets  of  information  include  an  advertisement  and  letter  of  application,  and 
information  associated  with  planning  for  a trip  to  Europe. 
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Since  there  is  no  way  of  knowing  for  certain  what  skills  have  been  applied  by 
a particular  student  or  group  of  students  when  answering  a particular 
examination  item,  results  have  not  been  reported  in  this  document  in  terms  of 
taxonomic  level.  Nonetheless,  taxonomic  level  was  an  important  design 
consideration  in  the  development  of  the  English  33  Diploma  Examination. 

Table  3 shows  how  many  items  from  each  reporting  category  have  been  placed 
within  each  taxonomic  level.  In  addition.  Table  3 gives  the  proportion  of  the 
total  examination  mark  assigned  to  each  taxonomic  level. 
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Table  3 

English  33  Ddploma  Examination  Blueprint 
Part  B:  Reading  (Multiple  Choice) 
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The  examination  presents  a variety  of  selections  from  fiction,  non-fiction  (literary  and  functional),  poetry,  and  modern  drama. 


CHAPTER  3 


ADMINISTRATION  OF  THE  EXAMINATION 


In  December  1983,  the  Student  Records  and  Computer  Services  Branch  of  Alberta 
Education  received  Confirmation  of  Registration  lists  from  principals  of  all 
high  schools  in  Alberta  with  students  registered  in  Grade  12.  Principals  were 
asked  to  verify  the  registration  of  students  in  diploma  examination  courses 
within  their  schools  and  to  indicate  any  changes  in  registration  that  might 
have  occurred.  These  verified  lists  allowed  Alberta  Education,  through  the 
Student  Evaluation  Branch,  to  estimate  numbers  of  examinations  required  for 
each  diploma  examination  course  for  each  school.  It  also  allowed  answer 
sheets  and  scoring  sheets  to  be  prepared  with  specific  student  and  school 
identification  information. 

Also  in  December  1983,  each  superintendent  and  high  school  principal  was  sent 
a copy  of  the  document  entitled  Guidelines  and  Procedures  for  Administering 
the  Grade  12  Diploma  Examinations:  January  1984  Administration.  This 

document  included  explanations  of  the  duties  and  responsibilities  of  the 
various  parties  involved  in  administering  the  examinations  (the  school 
jurisdiction,  the  school  principal  or  Chief  Presiding  Examiner,  and  the 
Presiding  Examiner).  In  addition,  information  about  security  of  the 
examinations,  confidentiality  of  students'  answers,  exceptional  cases,  and 
examination  rules  was  included. 

After  the  examinations  were  printed,  they  were  placed  in  school  packages  by 
subject  according  to  the  Confirmation  of  Registration  list  that  had  been 
verified  by  the  principal.  Each  school  was  sent  an  extra  number  of 
examinations  — approximately  10%  more  than  the  list  indicated.*  This  allowed 
for  changes  in  registration  and  provided  for  mature  students  — persons  at 
least  19  years  old  as  of  September  1,  1983,  who  had  been  out  of  school  for  at 
least  eight  consecutive  months  after  their  18th  birthday.  Mature  students 
were  allowed  to  write  any  diploma  examination  for  100%  of  the  final  mark 
without  being  currently  enrolled  in  the  course. 

A package  of  verified  copies  of  the  diploma  examinations  was  also  prepared  for 
each  Chief  Presiding  Examiner  (high  school  principal).  In  addition.  Chief 
Presiding  Examiner's  kits  (one  per  school)  and  Presiding  Examiner's  kits 
(approximately  one  for  each  30  students  in  the  school)  were  prepared.  The 
Chief  Presiding  Examiner's  kit  and  the  Presiding  Examiner's  kits  contained 
specific  instructions  necessary  to  ensure  reliable  administration  of  the 
examinations.  These  kits  also  contained  a copy  of  the  Examination  Rules. 

All  of  this  material  — the  packages  of  examinations,  the  verified  copies,  the 
Chief  Presiding  Examiner's  kits,  and  Presiding  Examiner's  kits  — was  sent  to 
the  central  office  of  each  school  jurisdiction  in  Alberta  in  early  January 
1984  for  dissemination  to  schools. 


*A11  high  schools  with  students  registered  in  Grade  12  received  a minimum  of 
three  extra  copies  of  each  diploma  examination  to  be  kept  on  file  in  the 
school  as  information  for  teachers  and  students. 
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School  jurisdictions  were  responsible  for  delivering  examination  materials  to 
schools  before  the  administration  dates.  In  the  case  of  the  English  33 
Diploma  Examination,  Part  A:  Written  Response  was  administered  between 

9:00  a.m.  and  11:30  a.m.  on  Tuesday,  January  17,  1984,  and  Part  B:  Reading 

(Multiple  Choice)  was  administered  between  9:00  a.m.  and  11:00  a.m.  on  Friday, 
January  27,  1984.  School  jurisdictions  were  responsible  for  keeping  the 
examinations  secure  prior  to  administration  and  for  supervising  the  collection 
of  examination  materials  from  schools  after  administration.* 

Examinations  were  returned  to  Alberta  Education  in  the  following  manner. 

School  jurisdictions  in  Zones  1,  4,  5,  and  6 returned  materials  as  directed  by 
their  Regional  Offices  of  Education.  Jurisdictions  in  Zones  2 and  3 returned 
materials  directly  to  the  Student  Evaluation  Branch  receiving  area  at  the 
Legislature  Annex,  9718  - 107  Street,  Edmonton.  At  the  Legislature  Annex,  the 
examinations  from  each  school  were  checked  against  their  respective  Lists  of 
Candidates  and  prepared  for  marking. 


*Grade  12  diploma  examinations  written  by  students  enrolled  in  category  I 
private  schools  were  administered  through  their  Acting  Superintendents  — 
usually  Regional  Office  of  Education  personnel. 


CHAPTER  4 


SCORING  OF  PART  A:  WRITTEN  RESPONSE 


Organization  of  Markers 

A marking  centre  was  established  in  Edmonton  on  the  fourth  floor  of  the 
Legislature  Annex.  Seventy-four  teachers  from  across  the  province  of  Alberta 
spent  approximately  one  week  (January  30,  1984  to  February  3,  1984)  marking 
Part  A:  Written  Response.  To  qualify  for  marking,  each  teacher  had  to  have  a 

valid  permanent  Alberta  teaching  certificate,  had  to  have  taught  English  33 
for  at  least  two  years,  and  had  to  be  currently  teaching  English  33. 

The  74  markers  were  divided  into  15  groups  (14  groups  of  5 and  1 group  of  4) 
to  facilitate  the  administration  of  marking.  Each  group  was  headed  by  a group 
leader  who  was  responsible  for  helping  to  train  the  group  of  markers  and  to 
conduct  reliability  reviews  (see  also  page  12).  The  15  group  leaders  and  six 
other  English  33  markers  were  brought  to  the  marking  centre  for  an  initial 
meeting  with  Student  Evaluation  Branch  personnel  on  Saturday,  January  28, 

1984,  to  respond  to  the  scoring  guides  (see  Appendix  A,  page  27)  that  had  been 
prepared;  to  read,  discuss,  and  come  to  a consensus  about  the  quality  of 
papers  selected  for  training  purposes;  and  to  participate  in  a standard- 
setting exercise. 


Training 

On  Monday,  January  30,  1984,  the  74  markers  met  at  the  Legislature  Annex. 

Most  of  the  first  morning  was  used  for  training  the  74  markers.  The  training 
began  in  plenary  session  with  an  overall  review  of  a marker's  manual,  a copy 
of  which  had  been  provided  for  each  of  the  markers.  The  criterion-referenced 
scoring  scales  for  Sections  I and  II  of  Part  A:  Written  Response  and  the  key 
to  Section  III  were  presented  to  the  markers.  Markers  then  regrouped  into 
their  15  small  groups  to  read  and  discuss  four  papers  that  had  been  selected 
for  training  purposes.  Group  leaders  led  the  discussion  and  later,  again  in 
plenary  session,  reported  on  the  opinions  of  their  groups.  Also,  during  this 
last  plenary  session,  the  mechanics  of  the  marking  procedure  (paper  flow  and 
filling  of  forms)  were  explained  to  the  markers. 


Marking 

The  remainder  of  the  week  was  used  to  evaluate  student  papers  independently. 

In  addition,  volunteers  remained  for  evening  marking  sessions  on  Wednesday, 
February  1,  1984,  and  Thursday,  February  2,  1984.  Volunteers  also  remained 
for  a "clean-up"  session  on  Saturday,  February  4,  1984. 

Before  papers  were  made  available  to  the  markers,  student  identification 
information  was  removed,  and  the  papers  were  placed  in  bundles  of  eight. 
Bundles  were  placed  in  five  batches  of  approximately  equal  size.  Batches  of 
papers  were  processed  one  at  a time  so  that  discrepant  papers  could  be 
identified  and  pulled  for  remarking.  When  marking  any  paper,  the  markers  had 
no  way  of  knowing  for  certain  the  identity  of  the  student  who  wrote  it,  or  the 
student's  school.  Markers  were  instructed  to  refer  to  their  group  leader  any 
paper  in  which  personal  knowledge  of  the  student  was  suspected. 
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All  papers  were  given  three  independent  markings.  A marker  would  take  a 
bundle  of  papers  from  a table  labelled  "Unmarked  Papers"  and  check  the  back 
cover  of  each  paper  in  the  bundle  to  make  certain  he  had  not  yet  read  any  of 
the  papers.  He  would  then  return  to  his  marking  station  and  independently 
mark  each  of  the  papers  in  the  bundle  by  applying  the  criterion-referenced 
scales  and  the  key  shown  in  Appendix  A.  The  marker  would  assign  a mark  of  1 
(low)  to  5 (high)  on  each  scale  to  be  applied  to  Sections  I and  II  of  the 
paper.  He  would  then  assign  appropriate  marks  for  content  to  the  three 
questions  in  Section  III  and  would  assign  a mark  for  Matters  of  Convention  for 
all  three  questions  in  Section  III  taken  together.  All  of  this  information 
was  placed  by  the  marker  on  a machine-scorable  sheet.  (See  Appendix  B, 
page  33.)  The  marker  would  place  his  marker  number  on  the  back  cover  of  each 
paper  in  the  appropriate  box  and  return  the  bundle  of  papers  to  another  table 
labelled  "Marked  Papers."  The  process  would  then  be  repeated. 


Reliability  Reviews 

Reliability  of  results  was  of  prime  concern  during  the  marking  sessions. 
Because  of  this,  reliability  review  sessions  were  conducted  twice  daily  — at 
10:00  a.m.  and  at  2:00  p.m.  At  these  sessions,  all  markers  were  given 
identical  papers  to  read  and  to  mark  independently.  Then,  the  group  leaders 
recorded  their  marks.  Each  group  then  discussed  the  two  papers  scale  by 
scale,  with  the  discrepant  or  divergent  markers  on  a particular  scale  speaking 
first.  At  the  end  of  the  reliability  review,  group  leaders  forwarded  their 
recorded  marks  to  Student  Evaluation  Branch  personnel,  who  then  tallied  all 
marks  and  posted  the  distribution  of  marks  for  that  session.  This  information 
provided  useful  feedback  for  monitoring  the  application  of  the  scoring  guides 
by  the  markers.  Group  membership  changed  at  each  of  the  eight  different 
reliability  review  sessions. 

As  a further  check  on  reliability,  markers  were  given  two  individual 
performance  reports  — one  on  Wednesday,  February  1,  1984,  and  one  on  Friday, 
February  3,  1984.  These  performance  reports  gave  information  on  numbers  of 
papers  marked  by  an  individual  and  the  percentage  of  these  papers  requiring 
rescoring  because  of  discrepancy.  The  reports  also  informed  individual 
markers  of  the  percentage  of  scores  on  each  of  the  scale  points  (0  to  5)  they 
were  assigning  for  each  scale.  In  addition,  markers  were  shown  the  frequency 
and  direction  of  each  discrepancy  on  each  scale.  Using  the  performance 
reports,  each  marker  could  compare  his  own  performance  with  that  of  the  group 
of  74  markers  as  a whole.  A sample  performance  report  is  presented  in 
Appendix  C,  page  34. 


Scale-by-Scale  Assignment  of  Final  Mark 

Once  each  of  the  papers  from  a batch  had  received  three  markings,  their 
scoring  sheets  (see  Appendix  B,  page  33)  were  scanned  and  the  results 
determined  by  a computer.  The  computer  was  programmed  to  assign  the  median 
mark  for  each  scale  as  the  final  mark. 

If,  for  example,  the  first  marker  had  awarded  a particular  paper  a 3 for 
Thought  and  Detail  on  Section  I,  the  second  marker  a 2,  and  the  third  a 4,  the 
computer  would  have  assigned  that  paper  the  median  mark  of  3 for  that 
particular  scale.  In  this  case,  the  marks  given  by  the  second  and  third 
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markers  (2  and  4)  would  have  been  considered  divergent,  since  they  each 
differed  from  the  median  mark  by  one  scale  point.  If,  on  the  other  hand,  the 
first  marker  had  awarded  a 4,  the  second  a 4,  and  the  third  a 2,  the  computer 
would  have  assigned  the  median  mark  of  4 for  the  scale.  In  this  second  case, 
the  marks  given  by  the  first  two  markers  would  have  been  in  agreement 
(both  4's),  but  the  mark  given  by  the  third  marker  (2)  would  have  been 
considered  discrepant,  since  it  differed  from  the  median  mark  by  more  than  one 
scale  point. 


Handling  of  Discrepant  Pape r s 

Discrepancy  on  one  scale  alone  was  not  enough  to  require  a fourth  marking  for 
any  paper.  If,  however,  a number  of  scales  on  a particular  paper  were 
discrepantly  marked  by  the  first  three  readers,  that  paper  was  considered  to 
be  discrepant  and  was  given  a fourth  marking.  Marks  assigned  during  the 
fourth  marking  had  to  fall  within  the  range  of  marks  that  had  been  established 
by  the  first  three  markers.  If,  for  example,  the  first  marker  had  awarded  the 
paper  a 1 on  a particular  scale,  the  second  marker  a 3,  and  the  third  marker  a 
3,  the  fourth  marker  could  award  the  paper  a 1,  a 2,  or  a 3 on  this  same 
scale.  The  fourth  marker  could  not  go  outside  the  set  range  of  marks  to 
assign  the  paper  a 0,  a 4,  or  a 5 on  that  scale. 
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CHAPTER  5 


RESULTS  AND  OBSERVATIONS 


Total  Examination  Results 

The  results  of  the  English  33  Diploma  Examination  are  summarized  in  Table  4. 
The  average  mark  for  all  students  for  the  total  examination  was  63.7%,  very 
close  to  the  target  mean  of  65.0%.  Students  achieved  an  average  of  32.3  marks 
out  of  50  on  Part  A:  Written  Response,  and  an  average  of  31.4  marks  out  of  50 

on  Part  B:  Reading  (Multiple  Choice). 

Table  4 


Total 

Examination 

Results 

Mean 

Standard  Deviation 

Total  Examination 

63.7% 

10.7 

Part  A:  Written 

Response 

32.3 

5.9 

Part  B:  Reading 

(Multiple  Choice) 

31.4 

6.4 

Standard  test  statistics  indicate  that  the  multiple-choice  section  has  a 
satisfactory  level  of  reliability  for  a test  measuring  a broad  range  of 
concepts  and  skills.  The  KR-20  coefficient  for  the  multiple-choice  portion  of 
the  examination  was  0.818. 


Standard-Setting 

Every  effort  was  made  to  design  an  English  33  diploma  examination  that  would 
be  a valid  and  reliable  measure  of  what  students  can  be  expected  to  know  as  a 
result  of  instruction  in  this  course.  A specific  standard  or  level  of 
expectation  inherent  in  the  examination  was  established  through  careful  test 
development  procedures.  To  ensure  that  each  form  of  examination  administered 
in  1984  will  be  parallel  to  each  other,  the  Student  Evaluation  Branch  has 
adopted  a process  of  standard-setting.  One  way  to  review  the  standards 
inherent  in  each  examination  is  to  involve  classroom  teachers  in  making 
judgments  about  the  difficulty  of  the  examination. 
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The  teachers  who  marked  the  written-response  portion  of  the  examination 
reviewed  the  difficulty  level  of  each  question  in  terms  of  a borderline 
passing  student  (who  merits  50%) , a borderline  "B"  student  (who  merits  65%), 
and  a borderline  "A"  student  (who  merits  80%) . After  teachers  gave  their 
initial  judgements  on  question  difficulty,  they  were  given  information  about 
the  actual  distribution  of  students'  examination  marks.  They  were  then  given 
the  opportunity  to  modify  their  judgements. 

In  English  33,  teachers  suggested  that  a slightly  larger  number  of  students 
should  receive  A's  and  B's  than  in  the  actual  distribution,  and  that  a 
slightly  smaller  number  should  receive  C's  and  F's.  However,  teachers' 
suggestions  were  so  close  to  the  actual  distribution  of  marks  in  English  33 
that  it  was  decided  to  accept  the  actual  distribution  as  the  standard. 


Relationship  Between  Examination  Marks  and  School -Awarded  Marks 

The  averages  for  examination  marks,  school-awarded  marks,  and  final  blended 
marks  were  63.7%,  59.1%,  and  61.9%  respectively.  The  percentages  of  students 
receiving  A's,  B's,  C's,  and  F's  is  given  in  Table  5. 

Table  5 


Percentage  Distribution  of  A's,  B's,  C's,  and  F's 


Letter 

Grade 

Examination 

Mark 

Schoo 1 -Awa  r ded 
Mark 

Final  Blended 
Mark* 

A (80-100%) 

5.7 

3.1 

2.1 

B (65-79%) 

43.4 

29.0 

36.8 

C (50-64%) 

42.9 

53.4 

55.0 

F (0-49%) 

8.0 

14.5 

6.1 

*These  final  blended  marks  include  the  marks  of  students  who  did  not  receive 
BOTH  a school -awarded  mark  and  an  examination  mark  for  various  reasons  (see 
pages  1 and  2 ) . 

The  correlation  between  examination  marks  and  school-awarded  marks  was  0.36, 
indicating  that  a relatively  weak  relationship  exists  between  these  two  types 
of  marks.  It  is  normal,  however,  to  expect  differences  between  the  two  marks 
because  the  examination  mark  is  limited  to  reflecting  achievement  only  in 
those  objectives  in  English  33  that  can  be  examined  by  a paper-and-pencil 
test,  while  the  school-awarded  mark  reflects  achievement  in  listening, 
speaking,  viewing,  and  all  other  course  objectives.  The  school-awarded  mark 
also  reflects  work  completed  by  the  students. 
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Results  for  Part  A:  Written  Response 


The  results  for  Part  A:  Written  Response  are  given  in  tables  6,  7,  and  8. 

The  three  tables  examine,  in  turn,  the  three  different  sections  of  Part  A. 

Section  I:  Personal  Response  to  Literature 

Table  6 

Percentage  Distribution  of  Scores  on  Section  I 


Score 

(Scale  Points) 

Reporting  Category 

Thought  and 
Detail 

Organization 

Matters  of 
Choice 

Matters  of 
Convention 

5 

(Proficient ) 

2.2 

2.5 

2.3 

5.1 

4 

(Capable) 

26.3 

33.8 

23.1 

36.1 

3 

(Adequate ) 

56.1 

56.6 

60.2 

42.5 

2 

(Limited) 

13.9 

6.3 

12.3 

12.9 

1 

(Poor) 

1.1 

0.3 

1.6 

2.9 

0 

( Insufficient ) 

0.5 

0.5 

0.5 

0.5 

Observations  regarding  the  results  for  Section  I:  Personal  Response  to 

Literature  are  as  follows:  For  Thought  and  Detail,  84.6%  of  the  students 

scored  at  an  Adequate  level  or  better  (received  scores  of  3,  4,  or  5),  and 
15.0%  of  the  students  scored  Limited  or  Poor  (received  scores  of  2 or  1).  For 
Organization,  92.9%  of  the  students  scored  Adequate  or  better,  and  6.6%  scored 
Limited  or  Poor.  For  Matters  of  Choice,  85.6%  of  the  students  scored  Adequate 
or  better,  and  13.9%  scored  Limited  or  Poor.  For  Matters  of  Convention,  83.7% 
of  the  students  scored  Adequate  or  better,  and  15.8%  scored  Limited  or  Poor. 
Only  0.5%  of  the  students  did  not  attempt  Section  I,  or  produced  compositions 
that  were  considered  to  be  insufficient  for  marking  purposes  or  were  off-topic 
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Section  II:  Functional  Writing 


Table  7 

Percentage  Distribution  of  Scores  on  Section  II 


Score 

(Scale  Points) 

Reporting  Category 

Thought  and 
Detail 

Matters  of 
Choice 

Matters  of 
Convention 

5 

(Proficient ) 

1.5 

1.0 

4.2 

4 

(Capable ) 

20.6 

17.2 

33.0 

3 

(Adequate ) 

56.2 

63.7 

44.7 

2 

(Limited) 

19.0 

15.4 

13.9 

1 

(Poor) 

1.6 

1.7 

3.2 

0 

( Insufficient ) 

1.1 

1.1 

1.1 

Observations  about  the  results  for  Section  II  are  as  follows:  For  Thought  and 

Detail,  78.3%  of  the  students  scored  at  an  Adequate  level  or  better  (received 
scores  of  3,  4,  or  5),  and  20.6%  of  the  students  scored  Limited  or  Poor 
(received  scores  of  2 or  1).  For  Matters  of  Choice,  81.9%  of  the  students 
scored  Adequate  or  better,  and  17.2%  scored  Limited  or  Poor.  For  Matters  of 
Convention,  81.9%  of  the  students  scored  Adequate  or  better,  and  17.1%  scored 
Limited  or  Poor.  Just  over  1.0%  of  the  students  did  not  attempt  Section  II, 
or  produced  compositions  that  were  considered  to  be  insufficient  for  marking 
purposes  or  were  off-topic. 
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Section  III:  Critical  Response  to  Visual  Communication 


Table  8 

Percentage  Distribution  of  Scores  on  Section  III 


Reporting  Category 

Score* 

Thought  and 

Detail 

Writing  Skills 

Question 

#1  Question 

#2  Question  #3 

Score 

(Scale  Points) 

Matters  of 
Convention 

5 

(Proficient) 

3.6 

4 

58.5 

4 

(Capable ) 

31.3 

3 

14.8 

46.4 

45.4 

3 

(Adequate) 

47.7 

2 

18.9 

40.9 

22.8 

2 

(Limited) 

13.8 

1 

3.6 

7.2 

14.9 

1 

(Poor) 

2.9 

0 

4.2 

5.5 

16.9 

0 

(Insufficient) 

0.7 

*Maximum  score  for  Question  #1  was  4;  maximum  score  for  Questions  #2  and  #3 
was  3. 


Observations  about  the  results  for  Section  III  are  as  follows:  For  Thought 

and  Detail  in  question  #1,  58.5%  of  the  students  scored  full  marks  (4  out  of 

4) ,  14.8%  scored  3 marks,  18.9%  scored  2 marks,  3.6%  scored  1 mark,  and  4.2% 
of  the  students  received  no  marks.  For  Thought  and  Detail  in  question  #2, 
46.4%  of  the  students  scored  full  marks  (3  out  of  3),  40.9%  scored  2 marks, 
7.2%  scored  1 mark,  and  5.5%  of  the  students  received  no  marks.  For  Thought 
and  Detail  in  question  #3,  45.4%  of  the  students  scored  full  marks  (3  out  of 
3),  22.8%  scored  2 marks,  14.9%  scored  1 mark,  and  16.9%  of  the  students 
received  no  marks.  For  Writing  Skills  (Matters  of  Convention)  in  Section  III, 
82.6%  of  the  students  scored  Adequate  or  better  (received  scores  of  3,  4,  or 

5) ,  16.7%  of  the  students  scored  Limited  or  Poor  (received  scores  of  2 or  1), 
and  0.7%  of  the  students  produced  writing  considered  to  be  insufficient  for 
marking  purposes . 
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Results  for  Part  B:  Reading  (Multiple  Choice) 


The  results  for  Part  B:  Reading  (Multiple  Choice)  by  reporting  category  are 

presented  in  Table  9.  It  is  important  to  note  that  two  items  (#  36  and  # 38) 
from  the  first  reporting  category.  Meanings,  were  deleted  from  Part  B.  This 
meant  that  the  50%  of  the  total  mark  for  the  English  33  Diploma  Examination 
assigned  to  Part  B was  calculated  from  a possible  maximum  raw  score  of  63. 
Discussion  of  items  36  and  38,  and  of  other  selected  items,  follows  later  in 
this  report. 


Table  9 

Results  for  Part  B:  Reading  (Multiple  Choice) 

(Raw  Scores) 


Reporting  Category 

Mean 

Standard 

Deviation 

1. 

Meanings  (23  items) 

13.6 

3.6 

2. 

Relationships  of  Form  and  Content  (8  items) 

4.5 

1.6 

3. 

Human  Experience  and  Values  (19  items) 

12.5 

3.0 

4. 

Life  Skills  (13  items) 

8.9 

1.9 

For  Meanings,  all  students  scored  an  average  of  13.6  items  correct  out  of  a 
possible  total  of  23,  or  59.1%.  For  Relationships  of  Form  and  Content, 
students  averaged  4.5  items  correct  out  of  a possible  total  of  8,  or  56.3%. 

For  Human  Experience  and  Values,  students  averaged  12.5  items  out  of  19,  or 
65.8%,  and  for  Life  Skills,  students  averaged  8.9  items  out  of  13,  or  68.5%. 

Although  performance  in  the  different  reporting  categories  shows  some 
variation,  these  scores  are  not  directly  comparable.  The  sets  of  items  that 
make  up  each  reporting  category  were  not  selected  to  be  equal  in  average  level 
of  difficulty.  Thus,  differences  may  be  due  to  variations  in  item  difficulty 
rather  than  in  student  performance.  In  combination  with  jurisdiction  results, 
however,  the  norms  can  be  used  to  detect  patterns  of  relative  strength  or 
weakness  in  achievement. 
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Item  response  frequencies  for  all  65  items  appearing  on  Part  B:  Reading 

(Multiple  Choice)  are  presented  in  Table  10. 

Table  10 

Item  Response  Frequencies 


Item 

Key 

Distribution  of 
Responses  in  X 

Item  Ke] 

Distribution  of 
Responses  in  X 

A 

B 

C 

D 

7 

A 

B 

C 

D 

1 

C 

18 

12 

59 

11 

34 

A 

66 

18 

14 

2 

2 

B 

9 

77 

6 

8 

35 

B 

7 

65 

12 

16 

3 

C 

3 

5 

88 

4 

36* 

D 

20 

8 

17 

55 

4 

B 

22 

67 

6 

5 

37 

C 

32 

16 

42 

10 

5 

B 

17 

69 

8 

6 

38* 

C 

10 

23 

32 

35 

6 

A 

89 

5 

2 

4 

39 

B 

18 

73 

3 

6 

7 

C 

6 

4 

86 

4 

40 

C 

10 

17 

39 

34 

8 

A 

76 

22 

1 

1 

41 

D 

18 

24 

2 

56 

9 

A 

43 

13 

34 

10 

42 

D 

20 

22 

30 

28 

10 

B 

9 

81 

4 

6 

43 

D 

17 

2 

4 

77 

11 

B 

14 

45 

21 

20 

44 

B 

16 

49 

6 

29 

12 

A 

58 

14 

8 

20 

45 

B 

14 

48 

25 

13 

13 

C 

21 

12 

66 

1 

46 

A 

51 

25 

8 

16 

14 

D 

8 

2 

7 

83 

47 

C 

2 

5 

92 

1 

15 

B 

13 

54 

6 

27 

48 

B 

1 

78 

1 

20 

16 

A 

72 

11 

8 

9 

49 

D 

5 

12 

28 

55 

17 

D 

13 

13 

12 

62 

50 

B 

7 

49 

6 

38 

18 

A 

59 

9 

19 

13 

51 

C 

4 

19 

75 

2 

19 

B 

14 

64 

9 

13 

52 

D 

2 

11 

60 

27 

20 

A 

79 

4 

2 

15 

53 

B 

7 

86 

3 

4 

21 

D 

2 

14 

2 

82 

54 

D 

7 

9 

27 

57 

22 

A 

94 

2 

1 

3 

55 

B 

6 

90 

3 

1 

23 

D 

20 

11 

7 

62 

56 

D 

17 

1 

19 

63 

24 

B 

1 

59 

14 

26 

57 

A 

72 

10 

4 

14 

25 

B 

4 

40 

3 

53 

58 

B 

5 

72 

6 

17 

26 

B 

13 

52 

5 

30 

59 

B 

7 

38 

19 

36 

27 

C 

13 

11 

51 

25 

60 

D 

3 

3 

14 

80 

28 

D 

10 

19 

9 

62 

61 

D 

16 

10 

21 

53 

29 

D 

18 

3 

10 

69 

62 

D 

29 

7 

5 

59 

30 

D 

5 

13 

9 

73 

63 

C 

19 

11 

46 

24 

31 

C 

15 

20 

61 

4 

64** 

C 

15 

27 

49 

8 

32 

B 

20 

47 

7 

26 

65** 

B 

15 

55 

19 

10 

33 

A 

26 

15 

50 

9 

N = 4804 

*Items  36  and  38  were  deleted  from  the  examination.  This  meant  that  the  50% 
of  the  total  examination  mark  contributed  by  Part  B:  Reading  (Multiple  Choice) 

was  calculated  out  of  a possible  raw  score  of  63.  Item  response  frequencies 
for  items  36  and  38  are  based  on  an  early  sample  (N  = 505). 

**Totals  do  not  add  to  100  because  approximately  IX  of  the  students  did  not 
respond  to  items  64  and  65. 

Totals  may  not  add  to  100  due  to  rounding. 
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Discussion  of  Selected  Items 


Item  1 pertained  to  a short  poem  by  David  Allan  Evans  entitled  "Neighbors." 

1.  The  speaker  suggests  that  the  woman's  facial  features  are 

A.  indistinct 

B.  flat 

C.  sharp 

D.  expressive 

Table  10  shows  that  18%  of  the  students  chose  alternative  A as  the  right 
answer,  12%  chose  alternative  B,  59%  chose  alternative  C,  which  is  the  keyed 
response,  and  11%  chose  alternative  D.  This  particular  item  was  of  average 
difficulty. 

Item  25  was  considerably  more  difficult.  It  pertained  to  an  excerpt  from  the 
play  For  Every  Man  an  Island , by  Helen  Porter. 

25.  This  play  is  intended  to  be  presented  as  a 

A.  television  program 

B.  radio  drama 

C.  movie  film 

D.  live  stage  play 

Four  per  cent  of  the  students  chose  alternative  A as  the  correct  answer,  and 
40%  chose  alternative  B,  which  was  the  keyed  response.  Three  per  cent  chose 
alternative  C,  and  53%  chose  alternative  D — more  than  had  chosen  the  keyed 
response.  It  appears  that  many  students  missed  the  sound  directions 
indicating  that  the  play  was  intended  to  be  produced  on  radio,  despite  the 
fact  that  explicit  stage  directions  in  the  text  of  the  play  referred  only  to 
sounds,  and  despite  the  fact  that  those  directions  about  sounds  were  printed 
in  capital  letters  and  double-spaced  so  that  they  would  be  clearly  visible. 

It  is  possible  that  students  did  not  refer  to  the  text  of  the  play  when 
answering  item  25. 

Item  33  also  proved  to  be  quite  difficult  for  students.  It  pertained  to  an 
article  entitled  "The  Country  Auction"  from  Harrowsmith  magazine. 

33.  One  seldom  finds  an  outstanding  bargain  at  an  auction  because 

A.  many  dealers  and  collectors  are  present 

B.  many  people  bid  on  the  same  items 

C.  the  auctioneer  knows  the  acceptable  price  of  each  item 

D.  true  value  is  very  rare  in  today's  economy 

Twenty-six  per  cent  of  the  students  chose  alternative  A,  which  was  the  keyed 
response,  15%  of  the  students  chose  alternative  B,  50%  chose  alternative  C, 
and  9%  chose  alternative  D.  This  item  discriminated  well,  however,  and  an 
examination  of  the  Harrowsmith  article  clearly  demonstrates  that  alternative 
A is  the  correct  response.  One  possible  explanation  is  that  students  did  not 
take  time  to  re-scan  the  article  before  answering,  and  chose  alternative  C 
because  it  seemed  a plausible  cause  for  the  effect  given  in  the  item's  stem. 


22 


Items  36  and  38,  also  pertaining  to  the  article  from  Harrowsmith,  were 
deleted  from  Part  B:  Reading  (Multiple  Choice)  before  students'  marks  were 
calculated.  Item  36  appeared  on  the  examination  as  follows: 

36.  To  get  a reasonable  bargain  at  an  auction,  a person  should 

A.  wait  for  the  lowest  price,  then  double  it 

B.  first  bid  a ridiculously  low  sum 

C.  let  the  bidding  drag,  then  raise  it  a dollar  at  a time 

D.  know  values  and  bid  confidently 

Although  this  item  was  only  slightly  more  difficult  than  average,  with  55%  of 
the  students  choosing  the  keyed  response  (alternative  D)  as  the  correct 
answer,  close  examination  of  the  item  statistics,  and  of  the  item  itself, 
revealed  that  the  item  contained  two  right  answers.  The  following  paragraph 
from  the  Harrowsmith  article  indicates  that  both  alternative  A and 
alternative  D are  correct  answers: 

"So  the  best  way  to  get  a good  deal  is  to  know  what  an  item  is  worth 
and  then  just  throw  your  bid  right  in.  Say  I start  the  bidding  at  $50; 
you  can  bet  that's  pretty  close  to  what  the  article's  worth.  Maybe  I'll 
have  to  drop  it  to  $20  before  someone  bids.  Soon  as  that  happens  throw 
in  your  bid  at  $40.  You'll  scare  people  away  and  still  get  a good  buy. 

If  I can  keep  the  bidding  going  up  a dollar  or  so  at  a time  I'll  have 

sold  the  item  for  $75  before  you  know  it.  . . ." 

The  auctioneer  states  that  "the  best  way  to  get  a good  deal  is  to  know  what  an 
item  is  worth  and  then  just  throw  your  bid  right  in."  In  other  words,  "know 
values  and  bid  confidently"  — alternative  D.  The  auctioneer  goes  on  to  say, 
however,  that  throwing  a bid  in  at  $40  immediately  after  someone  has  opened 
the  bidding  at  $20  will  "scare  people  away"  and  result  in  a "good  buy."  In 

other  words,  he  is  advising  auction-goers  to  "wait  for  the  lowest  price,  then 

double  it"  — alternative  A.  Item  36  was  deleted  because  of  the  two 
acceptable  answers. 

Item  38  appeared  on  the  examination  as  follows: 

38.  The  MAIN  purpose  of  this  essay  is  to 

A.  recreate  the  rural  atmosphere  of  auctions 

B.  analyze  the  psychological  make-up  of  auction-goers 

C.  offer  advice  about  auctions  to  potential  buyers 

D.  examine  the  character  of  a country  auctioneer 

Ten  per  cent  of  the  students  chose  alternative  A,  23%  chose  alternative  B,  and 
32%  chose  alternative  C,  which  is  the  keyed  response.  Alternative  D was 
chosen  by  35%  of  the  students.  In  addition,  this  item  discriminated  poorly. 

In  fact,  it  discriminated  against  those  students  whose  achievement  was  high  on 
the  other  items  in  Part  B.  This  item  was  intended  to  elicit  evaluation  skills 
and  was  constructed  to  be  a "best  answer"  rather  than  a "correct  answer" 
type.  However,  because  all  four  functions  mentioned  in  the  four  alternatives 
are  present  in  the  Harrowsmith  article,  and  because  the  right  answer  is  not 
clearly  defined,  students  had  undue  difficulty  with  this  question.  For  these 
reasons,  item  38  was  deleted. 
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Item  52  was  another  item  that  proved  to  be  difficult  for  the  students.  It 
pertained  to  a set  of  materials  on  planning  for  a trip  to  Europe,  and  appeared 
on  the  examination  as  follows: 

52.  Robin  wants  to  use  public  transit  systems  to  get  to  major  cultural 

attractions  in  large  cities.  Her  BEST  source  of  information  is 

A.  T 

B.  U 

C.  W 

D.  Y 

Two  per  cent  of  the  students  chose  alternative  A,  and  11%  chose  alternative 

B.  Neither  of  these  sources  mention  public  transit.  Sixty  per  cent  of  the 
students  chose  alternative  C,  which  does  not  mention  public  transit,  but  does 
contain  an  illustration  of  a tour  bus  as  part  of  an  advertisement  for 
conducted  tours  of  Europe.  Only  27%  of  the  students  chose  alternative  D,  the 
keyed  response,  which  had  the  words  "public  transportation"  included  in  the 
text . 

Many  of  the  items  on  Part  B were  relatively  easy.  Item  55,  for  example,  asked 
the  student  to  determine  the  meaning  of  a word  included  in  an  excerpt  from  the 
short  story  "Mrs.  Golightly  and  the  First  Convention,"  by  Ethel  Wilson.  It 
appeared  as  follows: 

55.  In  line  9,  the  narrator  describes  Mr.  Golightly  as  being  very 

"extroverted."  This  means  he  is 

A.  busy 

B.  outgoing 

C.  unthinking 

D.  nervous 

Six  per  cent  of  the  students  decided  that  the  word  "extroverted"  meant  "busy," 
and  chose  alternative  A.  Fully  90%  chose  alternative  B,  which  is  the  keyed 
response.  Three  per  cent  chose  alternative  C,  and  1%  chose  alternative  D. 

Two  factors  combine  to  make  this  an  easy  item.  First,  it  is  likely  that  many 
students  already  knew  the  meaning  of  the  word  "extroverted"  and  treated  the 
item  as  a discrete  question  — one  to  be  answered  without  reference  to  the 
excerpt.  Second,  the  sentence  in  which  the  word  appears  in  the  excerpt 
contains  strong  contextual  clues.  Students  who  were  uncertain  of  the  meaning 
of  the  word  "extroverted"  could  find  ample  reason  from  the  excerpt  to  choose 
the  keyed  response. 
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Summary  of  Observations 
Total  Examination 

Achievement  on  the  English  33  Diploma  Examination  was  generally  regarded  as 
acceptable  by  teachers  involved  in  the  examination  development  and  marking 
process.  The  actual  examination  mean  of  63.7%  closely  approximates  the  target 
mean  of  65.0%.  In  addition,  the  fact  that  only  8.0%  of  the  students  writing 
the  examination  scored  less  than  50%  suggests  that  students  are  successfully 
learning  those  concepts  and  skills  covered  by  the  English  33  Diploma 
Examination.  It  should  be  noted,  however,  that  there  were  few  high  achievers 
on  this  particular  examination.  Only  5.7%  of  the  students  scored  80%  or 
better. 


Part  A:  Written  Response 

Students  responded  well  to  the  writing  assignments,  often  with  fresh  and 
creative  ideas.  On  Section  I:  Personal  Response  to  Literature,  the  vast 

majority  of  students  (83.7%  to  92.9%)  scored  Adeguate  or  better  depending  upon 
the  reporting  category  being  examined.  On  Section  II:  Functional  Writing, 

again  the  majority  of  students  (78.3%  to  81.9%)  scored  Adequate  or  better 
depending  upon  the  reporting  category  being  examined.  On  Section  III: 

Critical  Response  to  Literature,  students  scored  well  on  Thought  and  Detail, 
with  45.4%  to  58.5%  receiving  full  marks  depending  on  the  question  being 
addressed.  Students  also  scored  well  on  Writing  Skills  (Matters  of 
Convention),  with  82.6%  scoring  Adequate  or  better.  Overall,  students 
performed  better  than  expected  on  the  written  section  of  the  English  33 
Diploma  Examination. 


Part  B:  Reading  (Multiple  Choice) 

Achievement  on  Part  B:  Reading  (Multiple  Choice)  was  about  as  expected. 

Depending  upon  the  reporting  category  or  subtest  being  examined,  means  vary 
from  56.3%  to  68.5%.  The  overall  mean  on  Part  B:  Reading  (Multiple  Choice) 

is  31.4  raw  score  marks  out  of  50.  This  is  an  acceptable  standard  of 
achievement . 
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APPENDIX  A 


ENGLISH  33  WRITTEN-RESPONSE  SCORING  GUIDE 
Scoring  Guide  for  Section  I:  Personal  Response  to  Literature 


Thought  and  Detail  (Curriculum  Concepts  1,  3,  4) 

5 PROFICIENT  writing  presents  a situation  that  has  been  effectively 
established.  Precise  details  about  actions  and/or  characters  are 
deliberately  chosen  and  add  clarity.  Retrospection  is  insightful  and 
mature . 

4 CAPABLE  writing  presents  a situation  that  has  been  clearly  and 

appropriately  established.  Specific  details  about  actions  and/or 
characters  are  well-defined  and  plausible.  Retrospection  is  perceptive. 

3 ADEQUATE  writing  presents  a situation  that  has  been  clearly  established. 
Details  about  actions  and/or  characters  are  clear,  but  tend  to  be 
general.  Retrospection  is  conventional  but  clearly  stated  or  clearly 
implied. 

2 LIMITED  writing  presents  a situation  that  has  been  vaguely  established. 
Inappropriate  or  unclear  details  about  actions  and/or  characters  are 
haphazardly  chosen.  Retrospection  is  obscure  or  uncertain. 

1 POOR  writing  presents  an  inappropriate  or  incomprehensible  situation. 
Details  about  actions  and/or  characters  are  unclear,  irrelevant,  or 
absent.  Retrospection  is  not  present  or  is  confusing. 

0 INSUFFICIENT  writing  is  of  insufficient  length  or  completely  off-topic, 
or  is  writing  in  which  the  assignment  has  not  been  attempted. 


Organization  (Curriculum  Concepts  2,  3,  4) 

5 PROFICIENT  writing  presents  an  introduction  designed  to  promote  further 
reading.  The  middle  is  developed  so  that  sentences  flow  smoothly  and 
coherently  to  an  appropriate  and  effective  conclusion. 

4 CAPABLE  writing  presents  an  effective  introduction.  The  middle  is 

developed  so  that  sentences  are  coherently  related.  The  conclusion  is 
appropriate . 

3 ADEQUATE  writing  presents  a functional  introduction,  a middle,  and  a 

conclusion.  Sentences  are  generally  related,  but  coherence  falters  on 
occasion. 

2 LIMITED  writing  lacks  either  an  introduction  or  a conclusion. 
Relationships  among  sentences  are  frequently  unclear. 

1 POOR  writing  lacks  an  introduction,  a conclusion,  and  coherence. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  the  organization  of  the  material. 
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APPENDIX  A (CONTINUED) 


Scoring  Guide  for  Section  I (continued) 

Matters  of  Choice  (Curriculum  Concept  3) 

5 PROFICIENT  writing  presents  a selection  and  use  of  words  and  structures 
that  is  accurate  and  generally  effective.  Words  are  correctly  used,  and 
many  sentences  have  been  deliberately  structured  for  effect. 

4 CAPABLE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  freguently  effective.  Words  are  seldom  misused, 
and  some  sentences  have  been  deliberately  structured  for  effect. 

3 ADEQUATE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  occasionally  effective.  Most  words  and 
structures  are  used  correctly,  but  attempts  to  structure  sentences  for 
effect  are  rare. 

2 LIMITED  writing  presents  a selection  and  use  of  words  and  structures  that 

is  frequently  inaccurate  and  ineffective.  Many  words  and  structures  are 

misused.  No  attempt  is  made  to  use  sentence  structure  for  effect. 

1 POOR  writing  presents  a selection  and  use  of  words  and  structures  that  is 

generally  inaccurate  and  ineffective.  Words  and  structures  are  misused  to 
such  an  extent  that  clarity  suffers. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  choice. 


Matter  of  Convention  (Curriculum  Concept  3) 

5 PROFICIENT  writing  is  essentially  free  from  errors  in  spelling, 

punctuation,  and  grammar.  These  errors  seldom  impede  the  communicative 
power  of  the  composition. 

4 CAPABLE  writing  has  occasional  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  seldom  impede  the  communicative  power  of  the 
composition. 

3 ADEQUATE  writing  has  several  errors  in  spelling,  punctuation,  and 

grammar.  These  errors  occasionally  impede  the  communicative  power  of  the 
composition. 

2 LIMITED  writing  has  frequent  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  often  impede  the  communicative  power  of  the 
composition. 

1 POOR  writing  has  errors  in  spelling,  punctuation,  and  grammar  that  are 
both  noticeable  and  jarring.  These  errors  severely  impede  the 
communicative  power  of  the  composition. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  convention. 
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APPENDIX  A (CONTINUED) 


Scoring  Guide  for  Section  II:  Functional  Writing 

Thought  and  Detail  (Curriculum  Concepts  1,  3,  4,  5) 

5 PROFICIENT  writing  presents  accurate  and  precise  information  enhanced  by 
interesting  and  imaginative  details  that  effectively  fulfil  the  purpose. 

4 CAPABLE  writing  presents  accurate  information  substantiated  by  interesting 
and  well-defined  details  that  efficiently  fulfil  the  purpose. 

3 ADEQUATE  writing  presents  essential  information  supported  by  sufficient 
detail  to  fulfil  the  purpose. 

2 LIMITED  writing  lacks  essential  information.  Supporting  details  are  scant 
and  haphazard  so  that  the  purpose  is  only  partially  fulfilled. 

1 POOR  writing  lacks  essential  information  and  supporting  details.  The 
purpose  is  not  fulfilled. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  thought  and  detail. 

Writing  Skills: 

Matters  of  Choice  (Curriculum  Concept  3) 

5 PROFICIENT  writing  presents  a selection  and  use  of  words  and  structures 
that  is  accurate  and  generally  effective.  Words  are  correctly  used,  and 
many  sentences  have  been  deliberately  structured  for  effect. 

4 CAPABLE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  frequently  effective.  Words  are  seldom  misused, 
and  some  sentences  have  been  deliberately  structured  for  effect. 

3 ADEQUATE  writing  presents  a selection  and  use  of  words  and  structures  that 
is  generally  accurate  and  occasionally  effective.  Most  words  and 
structures  are  used  correctly,  but  attempts  to  structure  sentences  for 
effect  are  rare. 

2 LIMITED  writing  presents  a selection  and  use  of  words  and  structures  that 

is  frequently  inaccurate  and  ineffective.  Many  words  and  structures  are 

misused.  No  attempt  is  made  to  use  sentence  structure  for  effect. 

1 POOR  writing  presents  a selection  and  use  of  words  and  structures  that  is 

generally  inaccurate  and  ineffective.  Words  and  structures  are  misused  to 
such  an  extent  that  clarity  suffers. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  choice. 
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APPENDIX  A (CONTINUED) 


Scoring  Guide  for  Section  II  (continued) 

Matters  of  Convention  (Curriculum  Concept  3) 

5 PROFICIENT  writing  is  essentially  free  from  errors  in  spelling, 

punctuation,  and  grammar.  Errors  that  are  present  do  not  reduce  the 
communicative  power  of  the  composition. 

4 CAPABLE  writing  has  occasional  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  seldom  impede  the  communicative  power  of  the 
composition. 

3 ADEQUATE  writing  has  several  errors  in  spelling,  punctuation,  and 

grammar.  These  errors  occasionally  impede  the  communicative  power  of  the 
composition. 

2 LIMITED  writing  has  frequent  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  often  impede  the  communicative  power  of  the 
composition. 

1 POOR  writing  has  errors  in  spelling,  punctuation,  and  grammar  that  are 
both  noticeable  and  jarring.  These  errors  severely  impede  the 
communicative  power  of  the  composition. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  convention. 
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APPENDIX  A (CONTINUED) 


Scoring  Guide  for  Section  III:  Critical  Response  to  Visual  Communication 

(Curriculum  Concepts  1,  3,  4,  5,  13,  14,  16,  17) 


1.  State  TWO  ideas  that  are  suggested  by  the  way  the  cartoonist  has  drawn  the 
first  taxpayer.  Refer  to  specific  details  in  the  cartoon  to  support  each 
of  your  ideas. 

Thought  and  Detail:  Two  marks  are  to  he  awarded  for  the  two  ideas 

suggested  by  the  way  the  cartoonist  has  drawn  the  first  taxpayer.  Two 
marks  are  to  be  awarded  for  identification  of  details  from  the  cartoon  to 
support  the  ideas  suggested . 


Possible  answers: 


1.  The  first  taxpayer  is  an  occupational  stereotype  and  represents 
farmers.  He  is  dressed  in  work  boots,  overalls,  a plaid  shirt,  and 
"cat"  hat. 

2.  The  cartoonist  is  suggesting  that  taxpayers  are  dishonest.  The 
first  taxpayer's  fingers  are  crossed  behind  his  back  showing  that  he 
is  lying.  His  halo  and  facial  expression  suggest  false  innocence. 
His  nose  reminds  us  of  Pinocchio. 

3.  The  first  taxpayer  is  worried.  Beads  of  perspiration  are  forming  on 
his  forehead. 

4.  The  first  taxpayer  lacks  confidence.  He  is  not  looking  the 
bureaucratic  beaver  straight  in  the  eyes. 

5.  The  first  taxpayer  has  a guilty  conscience.  He  is  sweating  and  is 
wearing  an  insecure  smile. 

Students  may  present  ideas  not  suggested  by  the  marking  guide. 

Each  answer  is  to  be  judged  on  its  own  merit. 


2.  Explain  how  you  would  feel  if  you  were  the  beaver.  Give  TWO  reasons  to 
support  your  answer. 

Thought  and  Detail:  One  mark  is  to  be  awarded  for  an  acceptable  answer 

to  the  question.  One  mark  is  to  be  awarded  for  each  of  the  two  supporting 
reasons . 

Possible  answers: 


The  beaver  might  feel  suspicious,  irritated,  angry,  upset,  concerned, 
incredulous,  disbelieving,  surprised,  shocked,  forceful,  powerful,  or 
amused. 

Students  may  identify  feelings  not  listed  above.  The  reasons  given 
should  clearly  support  the  feeling  described  in  answer  to  the  question. 
Each  answer  must  be  judged  on  its  own  merit. 
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APPENDIX  A (CONTINUED) 


Scoring  Guide  for  Section  III  (continued) 

3.  State  the  MAIN  idea  suggested  by  the  cartoon.  Refer  to  TWO  details  in  the 
cartoon  to  support  your  view. 

Thought  and  Detail:  One  mark  is  to  be  awarded  for  an  acceptable  main 

idea.  One  mark  is  to  be  awarded  for  each  of  the  supporting  details  from 
the  cartoon. 

Possible  answers: 


1.  Taxpayers  cheat  on  their  taxes.  All  of  the  taxpayers  in  the  cartoon 
are  dishonest;  all  have  long  noses,  and  some  have  falsely  innocent  or 
blatantly  guilty  expressions  on  their  faces. 

2.  The  country's  tax  system  is  unfair  and  unjust.  It  makes  taxpayers 
feel  like  criminals.  The  taxpayers  in  the  cartoon  have  been  made  to 
feel  like  liars  and  cheats  by  a suspicious  and  unfeeling  bureaucracy. 

Students  may  present  acceptable  answers  not  suggested  by  the  marking 
guide.  Supporting  detail  should  be  appropriate  to  the  interpretation 
chosen.  Each  answer  must  be  judged  on  its  own  merit. 


Writing  Skills 

Ten  marks  are  assigned  for  accuracy  and  clarity  of  expression.  These  marks 

are  to  be  awarded  on  the  basis  of  the  skills  shown  in  all  three  of  the 

guestions  in  Section  III.  The  scoring  guide  follows. 

Writing  Skills  (Matters  of  Convention) 

5 PROFICIENT  writing  is  essentially  free  from  errors  in  spelling, 

punctuation,  and  grammar.  Errors  that  are  present  do  not  reduce  the 
communicative  power  of  the  composition. 

4 CAPABLE  writing  has  occasional  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  seldom  impede  the  communicative  power  of  the 
composition. 

3 ADEQUATE  writing  has  several  errors  in  spelling,  punctuation,  and 

grammar.  These  errors  occasionally  impede  the  communicative  power  of  the 
composition. 

2 LIMITED  writing  has  frequent  errors  in  spelling,  punctuation,  and 
grammar.  These  errors  often  impede  the  communicative  power  of  the 
composition. 

1 POOR  writing  has  errors  in  spelling,  punctuation,  and  grammar  that  are 
both  noticeable  and  jarring.  These  errors  severely  impede  the 
communicative  power  of  the  composition. 

0 INSUFFICIENT  writing  is  so  deficient  in  length  that  it  is  not  possible  to 
assess  matters  of  convention. 
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